Papers with sentence generation
Deep Bayesian Natural Language Processing (P19-4)
Copied to clipboard
| Challenge: | Introduction to deep Bayesian learning for natural language addresses the fundamentals of statistical models and neural networks. |
| Approach: | This tutorial addresses the advances in deep Bayesian learning for natural language . it focuses on advanced Bayessian models and deep models . authors present case studies and domain applications to tackle different issues . |
| Outcome: | This tutorial focuses on advanced Bayesian models and deep models for natural language . case studies and domain applications are presented to tackle different issues in deep Bayessian processing, learning and understanding. |
Using Semantic Similarity as Reward for Reinforcement Learning in Sentence Generation (P19-2)
Copied to clipboard
| Challenge: | Existing models for sentence generation use cross-entropy loss as the loss function . however, cross-etropy is unable to evaluate sentences as a whole and lacks flexibility . et al., 2018: a novel approach to improve sentence generation models . |
| Approach: | They propose a method to train a model using estimated semantic similarity between output and reference sentences to alleviate cross-entropy loss problems. |
| Outcome: | The proposed model improves the BLEU scores from the baseline LSTM NMT model. |
Language Generation via Combinatorial Constraint Satisfaction: A Tree Search Enhanced Monte-Carlo Approach (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Generating natural language under complex constraints is a principled formulation towards controllable text generation. |
| Approach: | They propose a method to specify combinatorial constraints for sentence generation . they use a tree search algorithm embedded into the proposal process of the Markov Chain Monte Carlo . |
| Outcome: | The proposed method achieves consistent and significant improvement on multiple language generation tasks. |
Hybrid Semantics for Goal-Directed Natural Language Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing goal-directed natural language generation systems use first-order logic to represent semantics, but they are often slow due to the semantics of the partially realized text being checked. |
| Approach: | They propose to use logical semantics and distributional semantics to combine meaning representations to scale a goal-directed natural language generation system without losing expressiveness. |
| Outcome: | The proposed approach scales significantly better than the goal-directed generation system, but it is slower because the representations are not as precise as pure logical semantics. |
Generating Diverse Translations with Sentence Codes (P19-1)
Copied to clipboard
| Challenge: | Existing methods to generate diverse translations use different sentence structures . Xu et al., 2018: generating multiple valid translations with high diversity is difficult . |
| Approach: | They propose to use sentence codes to condition the sentence generation to obtain diverse translations . they propose to sample multiple candidates, each of which conditioned on a unique code . |
| Outcome: | The proposed method generates paraphrase translations with drastically different structures . the proposed method can be easily adopted to existing translation systems . |
Dynamic Topic Tracker for KB-to-Text Generation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing KB-to-text generation models suffer from an off-topic problem . existing models generate unrelated clauses regardless of input data . |
| Approach: | They propose a dynamic topic tracker that learns a global hidden representation for topics and recognizes the corresponding topic during each generation step. |
| Outcome: | The proposed model improves the performance of sentence generation and mitigates off-topic problem. |
Context-Interactive Pre-Training for Document Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Document machine translation typically suffers from a lack of document-level bilingual data. |
| Approach: | They propose a document machine translation model that incorporates contextual information into the training signals by capturing cross-sentence dependency within the target document and cross sentence translation to make better use of contextual information. |
| Outcome: | The proposed model outperforms baselines on three benchmark datasets and significantly outperformed previous approaches. |
Prior Knowledge and Memory Enriched Transformer for Sign Language Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for sign language translation do not explore all of them . visual and textual understanding and additional prior knowledge learning are challenging . |
| Approach: | They propose a method which integrates auxiliary information into vanilla transformer for SLT . they use visual-textual context information and additional auxiliary knowledge of a word . |
| Outcome: | The proposed method improves the understanding of sign language videos with visual and textual understanding and additional prior knowledge learning. |
ABCD: A Graph Framework to Convert Complex Sentences to a Covering Set of Simple Sentences (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work relies on rule-based methods dependent on parsing to identify atomic sentences. |
| Approach: | They propose a task to decompose complex sentences into simple ones . they propose atomic clauses as atomic sentences, and a graph edit task to predict edits . |
| Outcome: | The proposed model performs better than baselines on MinWiki and DeSSE. |
COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completion (2021.acl-long)
Copied to clipboard
| Challenge: | Existing inference models are opaque, but they can be made more interpretable by explicitly generating interim inference rules and using them to guide the generation of task-specific textual outputs. |
| Approach: | They propose a recursive inference framework that iteratively reads context sentences and dynamically generates contextualized inference rules, encodes them, and uses them to guide output generation. |
| Outcome: | The proposed framework generates better story sentences than baseline models, and is more interpretable than existing models. |
Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learning (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for training contrastive learning based sentence embedding models are largely influenced by the quality of sentence pairs. |
| Approach: | They propose a framework that decomposes LLMs into three stages for training . they propose to refine the generated content at these stages to ensure only high-quality sentence pairs are utilized to train a base contrastive learning model. |
| Outcome: | The proposed framework surpasses ChatGPT and ChatGPP in terms of performance. |
A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representations (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning disentangled representations of textual data are difficult to implement and suffer from the degeneracy of other losses in multi-class scenarios. |
| Approach: | They propose a variational upper bound to the mutual information between an attribute and the latent code of an encoder that controls the approximation error. |
| Outcome: | The proposed method is superior on fair classification and on textual style transfer tasks. |
Guiding Neural Machine Translation with Semantic Kernels (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Empirical studies show that our approach gains approximately an improvement of 1 BLEU score on most benchmarks over the Transformer baseline. |
| Approach: | They propose to extract several semantic kernels from a source sentence to capture global semantic information. |
| Outcome: | Empirical results show that the proposed approach improves 1 BLEU score on benchmarks . it is also 1.7 times faster than previous works on average at inference time . |
A Contrastive Cross-Channel Data Augmentation Framework for Aspect-Based Sentiment Analysis (2022.coling-1)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis is sensitive to multi-aspect challenges, resulting in multiple aspects in a sentence. |
| Approach: | They propose a framework that leverages an in-domain generator to construct more multi-aspect samples . they then boost the robustness of ABSA models via contrastive learning on these generated samples ." |
| Outcome: | The proposed framework outperforms baselines without any augmentations on accuracy and Macro- F1 . the proposed framework can generate more multi-aspect samples and boost the robustness of ABSA models . |
SuMe: A Dataset Towards Summarizing Biomedical Mechanisms (2022.lrec-1)
Copied to clipboard
| Challenge: | Biomedical studies often examine how one entity affects another in a biological context. |
| Approach: | They propose a biomedical mechanism summarization task that pairs biomedically relevant texts with their summaries. |
| Outcome: | The proposed task improves performance but produces acceptable outputs in 32% of instances. |
Memorization or Reasoning? Exploring the Idiom Understanding of LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | idioms have long posed a challenge due to their unique linguistic properties, which set them apart from other common expressions. |
| Approach: | They propose to use a large-scale dataset of idioms in six languages to evaluate LLMs' idiomatic processing ability. |
| Outcome: | The proposed model integrates contextual cues and reasoning to improve idiom understanding in LLMs, suggesting that their performance is influenced by memorization and reasoning. |